跳转至

面向长文本自动驾驶的规划对齐 Token 压缩

文章背景与核心概要

在自动驾驶领域，单一视觉-动作模型（Monolithic vision-action models）正逐渐成为一种新兴范式。然而，当编码长时序上下文以处理复杂的交互场景时，这种架构会产生庞大的 Token 序列，迅速超出实时计算的预算。尽管现有的 Token 压缩方法有助于缓解这一问题，但它们通常依赖于与实际驾驶行为脱节的基于规则的启发式方法（如时间衰减），从而面临丢失决策关键信息的风险。

为了解决这一痛点，本文提出了 COMPACT-VA，这是一个基于条件 VQ-VAE 的规划对齐工作记忆框架。该框架将压缩过程与历史轨迹以及从未来轨迹中蒸馏出的学习型规划意图相结合，在实现显著计算加速的同时，保留了至关重要的决策数据。闭环评估表明，与未压缩的处理方式相比，COMPACT-VA 实现了 3.3倍的加速 和 2.7倍的内存减少，在同等 Token 预算下的驾驶成功率提升超过 6%。

Planning-aligned Token Compression for Long-Context Autonomous Driving

arXiv ID: arXiv:2606.07464 [cs.RO]
Accepted by: IEEE Robotics and Automation Letters (RA-L) 2026
License: Creative Commons Attribution 4.0 International

作者

Authors

Zhixuan Liang, Yuxiao Chen, Yurong You, Peter Karkus, Wenhao Ding, Boyi Li, Alexander Popov, Yan Wang, Maximilian Igl, Yiming Li, Danfei Xu, Nikolai Smolyanskiy, Boris Ivanovic, Ping Luo, Marco Pavone

摘要

Summary

Monolithic vision-action models in autonomous driving often face severe computational bottlenecks when encoding long temporal contexts to manage complex interactions. While existing token compression methods help, they typically rely on rule-based heuristics (like temporal decay) decoupled from actual driving behavior, risking the loss of decision-critical information.

This paper introduces COMPACT-VA, a planning-aligned working memory framework built on a conditional VQ-VAE. By conditioning compression on historical trajectories and a learned planning intent distilled from future trajectories, COMPACT-VA preserves crucial decision-making data while achieving significant computational speedups. Closed-loop evaluations demonstrate a 3.3× speedup and 2.7× memory reduction over uncompressed processing, alongside a >6% improvement in driving success rates under comparable token budgets.

摘要

Abstract

Monolithic vision-action models represent an emerging paradigm in autonomous driving. However, this architecture produces token sequences that quickly exceed real-time computational budgets when encoding extended temporal context for complex interactions. While approaches like linear transformers and external memory try to make the context lightweight, token compression is most compatible with the architecture as it requires no backbone modifications.

Yet existing compression adopts rule-based heuristics like temporal decay, decoupled from planning, risking loss of decision-critical information. We propose COMPACT-VA, a planning-aligned working memory framework built on conditional VQ-VAE, compressing extended context into bounded representations. Compression is conditioned on both historical trajectory and a learned planning intent that the posterior encoder distills from future trajectories during training, while the prior encoder learns to predict it from compressed observations.

The compressed memory, concatenated with the predicted latent, feeds the policy for end-to-end optimization, planning with retained decision-critical information. We evaluate on high-signal dynamic scenarios where historical context is most critical for behavior correctness (e.g., stop, yield, or proceed), and accordingly design behavioral metrics. Under comparable token budgets, we achieve >6% improvement (68.3%) on success rates with consistent gains across metrics. Ablations validate planning-aligned coupling effectiveness. Closed-loop evaluation confirms that COMPACT-VA maintained general driving performance with 3.3× speedup and 2.7× memory reduction over uncompressed processing.

提交历史

Submission History

  • [v1] Fri, 5 Jun 2026, 17:16:21 UTC (1,198 KB)
  • [v2] Tue, 18 Aug 2026, 16:25:16 UTC (1,212 KB)
  • [v3] Wed, 19 Aug 2026, 16:12:12 UTC (1,213 KB) — Current version

全文与资源

Full-Text & Resources

license icon view license